DOCUMENT RESUME 



ED 383 286 



IR 017 141 



AUTHOR 
TITLE 

PUB DATE 
NOTE 



PUB TYPE 



Black, John B.; And Others 

Student Understanding and Learning from an 

Interpretation Construction Design. 

95 

7p.J In: Proceedings of the 1995 Annual National 
Convention of the Association for Educational 
Communications and Technology (AECT) , (17th, Anaheim, 
CA, 1995) ; see IR 017 139. 

Reports - Descriptive (141) — Speeches/Conference 
Papers (150) 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



MFOl/PCOl Plus Postage. 

Cognitive Development; Computer Networks; Cooperative 
Programs; '''Educational Environment; Higher Education; 
High Schools; '''Instructional Design; Knowledge Level; 
'''Multimedia Materials; '"Study Facilities; 
''Works tat i ons 

''Columbia University NY Teachers College; Dalton 
School NY; Study Support Environments 



ABSTRACT 

This study is a key concept in making design more 
fruitful in education. It is proposed that what students are doing 
when they construct knowledge is studying. For several years, 
Teachers College, Columbia University and the Dalton School (an 
independent school in New York City), have been collaborating on the 
Dalton Technology Project. The goal of the project is to use 
networked multimedia workstations to provide Study Support 
Environments (SSEs) . The core of study is the hermeneutic activity of 
constructing interpretations. From this perspective, the basis for 
cognition is interpretation based on background contextual 
information. In this paper, a framework for SSE design and its 
application to a specific SSE created as a part of the Dalton 
Technology Plan is described; and an evaluation that demonstrates its 
effectiveness is discussed. (Contains 11 references.) (Author/AEF) 
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Study is a key concept in making design more fruitful in education. We propose that what students 
are doing when they construct knowledge is studying. For several years, Teachers College, Columbia 
University and the Dalton School (an independent school in New York City), have been collaborating on 
the Dalton Technology Project. The goal of the project is to use networked multimedia workstations to 
provide Study Support Environments (SSEs). Creating SSEs allows us to create "a place for study in a 
world of instruction" (McClintock, 1971). The core of study is the hermeneutic activity of constructing 
interpretations. From this perspective, the basis for cognition is interpretation based on background 
knowledge and beliefs (Heidegger, 1962; Winograd and Flores, 1986). Thus, the key consideration in 
designing a SSE is fostering the construction of interpretations based on observations and background 
contextual information. In this paper, we describe a framework for SSE design and describe its application 
to a specific SSE created as part of the Dalton Technology Plan. After describing the SSE we report an 
evaluation that demonstrates its effectiveness. 

Interpretation Construction (ICON) Design Model 

Much of the effort in the Dalton Technology Project has gone into developing particular study 
systems for different subject areas, but we have also been working to specify what the appropriate design 
principles would be for this approach. As reported earlier (Black and McClintock, 1995; Black, McClintock 
and Hill , 1994) this constructivist design approach seems to be captured by the following seven principles 
comprising the Interpretation Construction (ICON) design model: 

1. Observation: Students make observations of authentic artifacts anchored in 

authentic situations 

2. Interpretation Construction: Students construct interpretations of observations and 

construct arguments for the validity of their nterpretations 

3. Contextualization: Students access background and contextual materials of 

various sorts to aid interpretation and argumentation 

4. Cognitive Apprenticeship: Students serve as apprentices to teachers to master 

observation, interpretation and contextualization 

5. Collaboration: Students collaborate in observation, interpretation and 

contextualization 

6. Multiple Interpretations: Students gain cognitive flexibility by being exposed 

to multiple interpretations 

7. Multiple Manifestations: Students gain transferability by seeing multiple 

manifestations of the same interpretations 

Some of these constructive design principles are adaptations from proposals by others. For 
example, the Cognitive Apprenticeship principle comes from Collins, Brown and Newman (1988), the 
Multiple Interpretations one from Spiro, Feltovich, Jacobson and Coulson (1992), and the Collaboration 
one from Johnson, Johnson, Holubec and Roy (1984). The Observation principle is a combination of 
recommendations by Brown, Collins and Duiguid (1989) and the Cognition and Technology Group at 
Vanderbilt (1990), but our focus on authentic artifacts is unique. Further, our emphasis on Interpretation 
Construction, Contextualization, and Multiple Manifestations is distinctive. 



An Example SSE 

To illustrate the application of this design framework, we describe an SSE program created for the 
Dalton Technology Plan. Specifically, we describe how these constructive design principles apply to the 
Galileo program used in 1 1th and 12th grade science (particularly for students not scientifically oriented) at 
the Dalton School. 
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In the Galileo program, students study astronomy and science in general by using observations of 
telescopic plates and a computer simulation of the sky to construct and test interpretations of astronomical 
phenomena. Students examine and make measurements on photographic plates from observatory 
telescopes and computer simulations of the sky (Observation), then relate these analyses to reference 
materials (Contextualization) containing what is know about astronomical objects (i.e., stars, planets, 
etc.). The teacher initially talks through how he would analyze and interpret examples of such astronomical 
data (Cognitive Apprenticeship) then the students form groups to work on some data 
(Collaboration), while the teacher coaches and advises them as they proceed. The students develop their 
own hypotheses and test them against the astronomical data (Interpretation Construction). Students 
defend their hypotheses using their analyses and reference materials both within and between the groups, and 
such argumentation together with background readings exposes them to various ways to interpret the data 
(Multiple Interpretations). As they proceed through the course, the students see how basic principles 
of astronomy, physics and chemistry can be used to make sense of different sets of astronomical data 
(Multiple Manifestations). 



An "valuation of Generic Skill Learning with the SSE 

Since we believe that interpretation is central to cognition and learning, we evaluated whether the 
Galileo program would increase students' interpretation skills. Specifically, we tested whether the students 
who had been through this program could make observations and interpretations in a completely new area 
better than students who had not been through the programs. For these studies, we chose an area unlikely 
to be familiar to precollege students ~ namely, cognitive psychology. 

In this evaluation study, the 1 1th and 12th grade students who had been through the Galileo 
program were compared to a comparable group on how well they could interpret and link three related 
cognitive psychology studies and their underlying principles. The students were given booklets containing 
descriptions of basic observations made in these three psychology studies together with various 
informational resources including relevant and irrelevant background material. The students were given 
three hours to perform the task and write a final report. These reports served as a measure of the students 
abilities to recognize particular patterns in the data, argue or explain the causes and effects of these patterns, 
as well as represent the data to support or refute their interpretations. 

Method 

Participants 

The experimental group consisted of 46 1 1th and 12th grade students at the Dalton School who were 
at the end of the Galileo course. The control group was 33 Dalton 10th and 1 lih grade students who planned 
to take the Galileo course during the following year. 



Materials 

The participants were provided with copies of data from three cognitive psychology studies. The 
participants were told that in all three of the studies they were reading about, the researchers select six 
students and have them memorize a list of 12 Subject- Verb-Object sentences (propositions). However, the 
studies differ in the following ways: 

Study One . One full day after remembering the list, participant memory is tested with items on 
computer screen, the propositions appear in the same form as they were memorized. The 
computer records how long it take each participant to respond with an affirmative answer. 

Study Two . A different set of six students are asked to recall the propositions they memorized a 
day before; however, this time, they are tested with the passive voice version of the sentences 
instead of the active voice version they memorized. Once again, the computer records how long it 
takes each participant to respond with an affirmative answer. 

Study Three. Yet another set of six students are asked to recall the propositions they memorized 
earlier; however, in this study, participants are tested with paraphrases of the sentences 
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memorized and asked to respond "yes" if the test sentence as essentially the same meaning as the 
memorized sentence. 

Upon reading these three studies and the background materials provided, Dalton students were asked 
to interpret the data using background materials provided as well as other outside sources. The background 
readings included both relevant information about information processing systems as well as propositional 
network theory. Irrelevant information from philosophy was also included as a distractor. 

Procedure 

Administering the Materials and Collecting Student Reports. 

The study was conducted in one 3-hour session. First, the experimenter passed out the assignment 
booklets and the teacher read the instructions on the first page of the assignment booklet. After the 
instructions, the students proceeded to work on the assignment in their groups. While doing the 
assignment the students were tree to use any of the resources in the Dalton School building (computers, 
libraries, etc.) including asking the experimenter clarification and information questions (the same 
experimenter and teacher conducted all sessions). At the end of the 3 -hour period the students handed in 
tiieir reports and all the work they had done in folders. 

Analysis of Student Reports 

The file folders from both the Galileo and prs-Galileo groups were evaluated along the following 
there dimensions: pattern recognition, explanation and argumentation, and data representation. These 
dimensions were weighted 2:3:1 based upon levels of difficulty. More specifically, groups could earn up to 
20 points for pattern recognition, 30 points for explanation and argumentation, and 10 points for data 
representation. Extra credit points were awarded for plausible recommendations for follow-up studies on the 
cognitive psychology principles tested; however, a majority of the students did not address this aspect of the 
data. 

Using our coding system, we were able to weigh answers in terms of difficulty as well as 
plausibility, the "optimal responses for pattern recognition, explanation and argumentation, and data 
representation are as follows: 

Pattern Recognition 

Study One. Students should recognize that the response time increases with 
the number of propositions per subject. For example, if lawyer has e propositions and 
doctor has 2 propositions then subjects will take longer to remember a proposition about 
the lawyer than the doctor. They are also expected to report the means. Partial credit is 
given for alternative patterns. Maximum Points: 4. 

Study Two . Students should note that the positive relationship between number 
of propositions per subject and response time still holds. They should also recognize that 
study two takes longer than study one. Again, means should be reported and partial credit 
is given for alternative patterns. Maximum Points: 7. 

Study Three . Students should recognize that the positive relationship between 
number of propositions per subject and response time still holds. They should also note 
that study three takes longer than study one and study two. Ideally, students should note 
that study three has a steeper climb (i.e., slope) than the other two studies. Means should 
be reported and partial credit is given for alternative patterns. Maximum Points: 9. 

Explanation an d Argumentation 

Study One . In this study, the optimal respones would relate the information 
processing and/or propositional network theories to the pattern recognized. Partial credit 
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was given for responses favoring individual differences. Testing hypotheses was also 
credited. Maximum Points: 6. 

Study Two . Students were supposed to relate the increase in mean response time 
to the stage of transforming a proposition from passive to active voice. They should have 
also noted that it takes time to match the transformed proposition to the database. 
Again, students were given points for appropriately using information processing theory 
or propositional network theory to support their hypotheses. Partial credit was given for 
responses favoring individual differences. Regardless of final outcome, hypothesis testing 
was also given partial credit. Maximum Points: 12. 

Study Three. Students were given credit for discussing how the overall increase 
in mean response time is due to an increase in search time, which is caused when a 
synonym for the original proposition is put in the recall test. They also should have 
noted that the increased time is due to the time required to match the synonym against the 
database. Students were given credit for using relevant background readings. Partial credit 
was given for responses favoring individual differences. Students were also given partial 
credit for reporting their hypothesis test and results. Maximum Points: 12. 

Data Representation 

Groups were given credit for creating spreadsheets that reported means and/or 
ratios, bar graphs that appropriately represented the data, propositional network diagrams, 
and alternative ways of representing the data to support explanations of trends. Ideally 
students would have been creating graphs that reflected the differences between studies. 
By plotting these trends, students would have recognized that the intercepts of study one 
and study three are the same but their slopes are different. Additionally, the intercept of 
study one and study two are different but their slopes are the same. Representations of 
these trends were likely to have improved both partem recognition and argumentation 
scores. Maximum Points: 10. 

Results 

Table 1 presents the results of the data analyses. As the first column in Table 1 shows, in total the 
Galileo group scored 33% higher than the pre-Galileo group (26.1 vs. 19.6 - out of 60 possible), and this 
is statistically a very significant difference, f(77)=4.56, p<.00l. 

Table 1 

Quantitative Analysis of Reports Written by Students 
in the Galileo Group and the Control Group 

Total Pattern Explanation & Data 

Recognition Argumentation Representation 

Galileo Group 26.1 9.8 14.2 2.1 

Control Group 19.6 7.7 11.8 0.1 

This significantly superior performance of the Galileo group also occurred in all three of the 
component scores: namely, in Pattern Recognition (9.8 vs. 7.7 -- out of a possible 20), f(77)= 2.40, p<.01 ; 
in Explanation and Argumentation (14.2 vs 1 1.8 -- out of a possible 30), f(77)=1.69, p<.05; and 
particularly in Data Representation where the pre-Galileo control group effectively got 0 (specifically, 0.1) 
whereas the Galileo group got 21% of the possible (2.1 out of 10 possible), f(77)=6.14, p<.001. 
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Conclusions 



We have proposed an approach to constructivist design (ICON) that makes interpretation 
construction of authentic artifacts in the context of rich background materials the central focus. We have 
shown how this approach can be applied to design a Study Support Environment for teaching science and 
scientific reasoning to students not scientifically oriented We have also shown that in addition to learning 
specific content, students using these programs acquire generalizable interpretation and argumentation skills. 
Thus, our constructivist design framework is useful both for guiding design and for producing valuable 
learning results. 
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